Learning and optimization of novel sensorimotor feedback loops: Internal models meet classical conditioning

We investigated how novel sensorimotor feedback loops can be formed in the course of learning. More specifically, we examined motor adaptation in an experiment which systematically paired a lateral force pulse at movement onset with a delayed visual target perturbation. Learning in this context means associating the cue and the perturbation, such that sensory feedback about the cue triggers a corrective action suitable for the upcoming perturbation. The data from the experiment reveals that human subjects gradually embraced the information content of the force impulse and used it to predict the forthcoming target displacement. Behaviorally adaptation manifested itself as (1) movements towards the anticipated target position (after the force pulse ended and before the target perturbation occurred), (2) reduction of the counter-productive stretch-reflex-like response to the force pulse, and (3) reduction in grip force (without change in arm impedance). To model the main effects of our study, we developed an extension to optimal control which uses a hedging approach to mix target-specific optimal feedback controllers weighted by an agent's belief in the plausibility of future goal outcomes. Using this method we accurately modeled the movement trajectories in the different phases of learning and observed that subject's beliefs converged to the true task statistics. We believe that our extension to optimal control is applicable to other tasks where the central nervous system (CNS) needs to maintain multiple hypotheses about future goals under consideration and prune them in an online fashion as novel information becomes available.